Configuration Tools: Working Together

Since the LISA conferences began, the character of a typical ‘‘large installation’’ has changed greatly. Most large sites tended to consist of a comparatively small number of handcrafted ‘‘servers’’ supporting a larger number of very similar ‘‘clients’’ (which would usually be configured with the aid of some automatic tool). A modern large site involves a more complex mesh of services, often with demanding requirements for completely automatic reconfiguration of entire services to provide fault-tolerance. As these changes have happened however, the tools available to provide configuration management for a site have not evolved to keep pace with these new challenges. This paper looks at some of the reasons why configuration tools have failed to move forward, and presents some suggestions for enabling the state of the art to advance. Background and Motivation Configuration Tools have been an important theme at LISA for many years, and most conferences include one or more papers in this area. Despite increasing recognition of the importance of the configuration problem, there remains both a lack of conceptual commonality and a lack of progressive innovation in the area. No significant standards have so far emerged, and new tools often merely espouse variations on existing approaches, frequently using both completely new specifications and code. For example, ten years separate the initial description of LCFG [5] and the presentation of Newfig [12], yet there is no clear evidence of conceptual progress. There is little or no attempt to create standards in a way which would reduce the barrier to new development or enable a greater shared understanding of the configuration problem. Wi t h i n the configuration management community, a great deal of ongoing discussion revolves around this apparent failure to either successfully disseminate the concepts underpinning existing tools, and the experiences gained from those tools, or to realise them in tool implementations suitable for a wider audience. Furthermore, despite the slow rate of progress, no sign of convergence between tools is apparent. There are essentially no standards in this area, with each tool being not only entirely coded from scratch, but also unable to interoperate with other tools, or share configuration data with them. Although the Configuration Description, Deployment and Lifecycle Management (CDDLM) working group of the Global Grid Forum (GGF) has published a standard interface for configuration tools on grid fabrics, their focus is upon exposing a web-service interface to the underlying configuration system, which falls somewhat wide of the mark in addressing the current problems faced by system administrators. This absence of standard tools has lead to many large sites developing their own tools. To create a functional configuration management system in its entirety, however, is a significant undertaking, and even those sites which have been able to invest sufficient resources in a system to make it sustainable have been unable to gain a wider community of users. It seems likely that in many cases, the sheer difficulty of developing and maintaining a large monolithic system (often written by people with a shortage of time and no background in software development) has limited both the functionality and the portability of tools. It is surprising indeed, given the importance of this area, that we find ourselves typically unable to recommend any current system to interested users. Levels of Configuration Perhaps because of the above difficulties, most existing tools deal with configuration specifications at a very primitive level; they involve ‘‘files’’ and ‘‘permissions’’, rather than ‘‘high-level’’ concepts such as ‘‘services’’, and the relationships between nodes – Figure 1 shows some typical statements at increasingly higher levels of abstraction. Ultimately, the requirements for a service are always expressed in high-level terms and most tools require these to be manually translated first into some lower-level requirements. As systems become more complex, this process is an increasing source of errors. Manual intervention is also unacceptable for autonomic systems which must reconfigure automatically in response to demand and failure. A future generation of configuration tools will need to be able to accept high-level requirement specifications, and to reason in much more sophisticated ways in order to determine the appropriate details. This is an inevitable consequence of the growing complexity of the modern installation, and the fact that the number of administrators a site needs does not scale linearly with the number of machines managed. There is an analogy with computer programming here: early languages provided very little abstraction from the underlying machine, but as the size and 19th Large Installation System Administration Conference (LISA ’05) 31 Configuration Tools: Working Together Anderson and Smith complexity of systems has grown, so has the separation from the hardware operations. In a similar way, configuration tools must adapt to aid administrators to manage the complexity of their systems. Low Level ‘‘Copy this disk image onto these machines’’ ↓ ‘‘Put these files on these machines’’ ↓ ‘‘Put this line in sendmail.cf on this machine’’ ↓ ‘‘Configure machine X as a mail server’’ ↓ ‘‘Configure machine X as a mail server for this cluster ’’ (and the clients will automatically be configured to match) ↓ ‘‘Configure any suitable machine as a mail server for this cluster’’ (and the clients will automatically be configured to match) ↓ Configure enough mail servers to guarantee an SMTP response time of X seconds High Level Figure 1: Levels of configuration specification. The Configuration Tool Zoo There are a plethora of tools available which purport to address some aspect of the configuration management problem. Many of these are described in more detail in surveys such as [6]. However, we present the following examples as typical, in that they demonstrate a variety of the most common approaches. In particular we note that: • The tools have different lexicons – for example, there is no common agreement on how to specify the ‘‘mail relay’’ for a particular client. • They use different languages and file formats. • Their deployment operations provide different guarantees regarding preand post-conditions. In general, these tools are also intended for direct manual use and present no intermediate interfaces between the human-generated specification, and the direct manipulation of the target machine.